BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth

摘要

We introduce a novel method for 3D object detection and pose estimation fromcolor images only. We first use segmentation to detect the objects of interestin 2D even in presence of partial occlusions and cluttered background. Bycontrast with recent patch-based methods, we rely on a "holistic" approach: Weapply to the detected objects a Convolutional Neural Network (CNN) trained topredict their 3D poses in the form of 2D projections of the corners of their 3Dbounding boxes for the pose of objects' parts. This, however, is not sufficientfor handling objects from the recent T-LESS dataset: These objects exhibit anaxis of rotational symmetry, and the similarity of two images of such an objectunder two different poses makes training the CNN challenging. We solve thisproblem by restricting the range of poses used for training, and by introducinga classifier to identify the range of a pose at run-time before estimating it.We also use an optional additional step that refines the predicted poses forhand pose estimation. We improve the state-of-the-art on the LINEMOD datasetfrom 73.7% to 89.3% of correctly registered RGB frames. We are also the firstto report results on the Occlusion dataset using color images only. We obtain54% of frames passing the Pose 6D criterion on average on several sequences ofthe T-LESS dataset, compared to the 67% of the state-of-the-art on the samesequences which uses both color and depth. The full approach is also scalable,as a single network can be trained for multiple objects simultaneously.